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DR K-12 Targeted Study: English Language Learners 


Introduction 

Although educational leaders, policy makers, and researchers have long emphasized the importance 
of science, technology, engineering, and mathematics (STEM) for the country's continued 
prosperity, increasing participation in STEM has remained a challenge for both the education and 
scientific communities (Pearson & Fechter, 1994; National Academies, 2007; President's Council of 
Advisors on Science and Technology, 2010; American Association for the Advancement of Science, 
1989). A historic imbalance in STEM participation persists, whereby proportionately fewer female 
and minority students, enroll in STEM courses and seek employment in STEM professions (National 
Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2010). 


Among those underrepresented in STEM are individuals who are English Language Learners (ELL), a 
sizable subgroup (5.3 million) among the country's elementary and secondary (PK-12) students; a 
subgroup whose growth is outpacing that of the overall PK-12 student population (The National 
Clearinghouse for English Language Acquisition and Language Instruction Educational Programs, 
2010; Aud et al, 2010). 


The difference between the academic performance of English-proficient and ELL students is 
substantial. For example. National Assessment of Educational Progress (NAEP) math results in 2005 
revealed that nearly half (46%) of ELL 4*'^ graders scored below basic in math as compared to 11 
percent of their white, 40 percent of their Black and 33 percent of their Flispanic peers. As students 
age, the gap widens: 71 percent of 8**^ grade ELL students scored below basic in math as compared 
to 21 percent of white, 59 percent of Black, and 50 percent of Flispanic 8*^ graders (Fry, 2007). The 
NAEP results in science are equally disparate: in 2005, 72 percent of 4**^ grade ELL students scored 
below basic while the same was true of 29 percent for non-ELL students. Likewise, 88 percent of 8*^ 
grade ELL students scored below basic compared to 39 percent for non-ELL students (Grigg, Lauko, 

& Brockway, 2006). 

Flistorically the subject area instruction of ELLs in math and science has received limited attention by 
the research community, which has instead focused on English proficiency and literacy of ELLs (Lee, 
2005). Thus, the research fields of math and science education have developed independent of the 
research field involving ELLs. Flowever, this trend is beginning to fade as science and math education 
researchers and their ELL education peers have begun to crossover to one another's domains to 
produce new understandings and strategies in math and science education specific to the needs of 
English Language Learners. 
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ELL and the National Science Foundation's Discovery Research K-12 Program 

For its part, the National Science Foundation's (NSF) Discovery Research K-12 (DR K-12) program is 
contributing to the growth of knowledge in these areas of crossover by funding ELL math and ELL 
science education projects that seek to understand and improve STEM learning and STEM 
instruction with ELLs. Across DR K-12's first three cohorts of awardees, the program supported 18 
such projects (11 in math education and 7 in science education). These projects involve 41 
investigators— principal investigators (Pis) and co-PIs— who are seeking to develop new knowledge, 
education approaches, and educational materials for ELL students and their teachers in math and 
science. 

At the DR K-12 Annual PI meeting in 2009, the first ELL-affinity group session was held to initiate a 
discussion of how DR K-12's Community for Advancing Discovery Research in Education (CADRE), a 
network supporting the efforts of the DR K-12 researchers, might facilitate their projects' success. 

An informal poll taken during the session produced a series of hypotheses describing the role DR K- 
12 is playing in extending the lines of inquiry in ELL and science and math education in important 
ways. Participants' responses suggested that the DR K-12 projects may be addressing different 
questions than what the fields' have traditionally pursued; they may be more likely to be conducted 
by researchers who possess expertise in math and/or science education than ELL; they may be more 
likely to be large-scale and involve more than one school district, and may even collect data from 
multiple states; and may be more likely to focus on secondary education rather than elementary 
education. Should these hypotheses hold true, the session's attendees concluded that the DR K-12 
projects have the potential to make a considerable contribution to the knowledge base on math and 
science education of ELLs. The group then asked CADRE to investigate and assess the merit of these 
hypotheses. 

To investigate the contribution of the DR K-12 portfolio to the knowledge base on math and science 
learning among ELLs, CADRE designed a study to explore the ELL work that is being conducted in the 
DR K-12 projects. This paper summarizes the work of this study. It begins with a description of the 
methodology employed, followed by a presentation of the findings, and finally a discussion of the 
conclusions drawn from this work. The findings are organized into discussions about the expertise 
held by the DR K-12 ELL researchers and research teams, the characteristics of the ELL research 
being conducted by the DR K-12 projects, and a comparison of the research conducted by the DR K- 
12 projects with published research on ELL-science education and ELL-math education. 

Methodology 

Building on the hypotheses generated during the ELL-affinity group session at the 2009 DR K-12 PI 
meeting, this study investigated the following specific research questions: 

1. What is the expertise of the investigators working on the DR K-12 ELL projects? 

2. What are the key characteristics and objectives of the DR K-12 ELL projects? 

3. How does the DR K-12 portfolio compare with the work typical of the larger fields of ELL 
math education and ELL science education? 
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To answer these questions, we performed content analyses of the DR K-12 projects' proposals and 
the curricula vitae (CVs) of their Pis and co-PIs. We also conducted structured literature searches for 
research on ELL and math education and updated an existing review of ELL and science education 
(Lee, 2005) widely considered a seminal synthesis of the field. Finally, we compared the results of 
the portfolio analysis with our review of the fields to consider how the DR K-12 ELL portfolio might 
be distinguished from the larger fields of research. 

CV Analysis 

The first research question was addressed through an analysis of investigators' CVs. We searched 
the Internet for the CVs of Pis and co-PIs, and directly requested copies in cases where we could not 
find them; we were able to secure CVs for all 41 researchers. Occasionally, researchers had websites 
with pertinent information (e.g., course listings) that we used to supplement the information in their 
CVs. Eleven of the CVs we obtained were not dated; of those that were, approximately more than 
half were from 2010 and a third from 2009; two additional CVs were from 2008, and two from 2006. 
It is important to note that some CVs contained more information than others and varied in length 
between 2 and 56 pages. Our analysis was based on the assumption that researchers include their 
most prominent work (publications, conferences, courses, and grants), which in turn would speak to 
their most salient interests and areas of expertise. 

We reviewed the CVs for indicators of content expertise across five proxies: field of highest degree, 
articles published in peer-reviewed/refereed journals (since 2000), grants awarded (since 2000), 
papers presented at conferences (since 2000), and courses taught. For four of the five proxies, data 
were available for more than three-quarters of the researchers. Exhibit 1 displays the number and 
percentage of researcher CVs coded for each of the proxies. 


Exhibit 1 : Avaiiabiiity of Data on Expertise Proxies for the CV Anaiysis 


Expertise Proxy 

N (%) All 
Researchers 

N {%) Pis 
Only 

Field of highest degree 

41 (100%) 

18 (100%) 

Peer-reviewed articles (since 2000) 

39 (95%) 

16 (89%) 

Grants awarded (since 2000) 

36 (88%) 

16 (89%) 

Conference papers (since 2000) 

32 (78%) 

12 (67%) 

Course lists* 

23 (56%) 

10 (56%) 


*9 researchers (6 Pis, 3 co-PIs) are not professors at universities, as indicated by their current title 
and/or employer. 


We coded the degree fields and the titles of presentation papers, peer-reviewed articles, grants, and 
courses according to the following substantive categories: ELL-language and literacy (ELL/LA); 
cultural and discursive studies in education; equity in education; math education; science education; 
other education; mathematics; science (including computer science and psychology not specified as 
educational psychology); and other. To identify each researcher's expertise evidenced within each 


Abt Associates Inc. 


Math and Science Education with Engiish Language Learners 


4 


proxy, we coded each entry according to the categories specified above, then counted the number 
of entries (e.g., classes, peer-reviewed articles, grants, and conference papers included on the CVs) 
and the number of times a given category was indicated; we credited a researcher with 
demonstrated expertise in an area when at least a quarter of his/her entries within a proxy were 
coded in a category. Finally, we created a composite indicator of expertise for each researcher by 
looking across all proxies for evidence of demonstrated expertise in each substantive category. 

Portfolio Analysis 

The second research question was addressed through a content analysis of project materials. To 
analyze the substance of the work conducted by the projects in the DR K-12 portfolio, we uploaded 
into NVivo (a software package that facilitates the coding and retrieval of relevant content) the 
project abstracts, proposal narratives, responses to reviewer questions, and annual reports that the 
projects had shared with CADRE. The materials were coded in each of the following areas: research 
goals and priorities, intellectual merit, broader impact, research rationale, research approach, ELL 
component, and researcher expertise. The content of each coded category was then analyzed. 

Literature Searches 

The third research question required a comparison of the DR K-12 portfolio with the larger field of 
published research, necessitating literature searches to find peer-reviewed published articles 
involving ELLs and math and/ or science education. We searched the ERIC and EBSCO databases^ 
using the following terms: "math" or "math educat*"^ or "science" or "science educat*" in 
combination with "English Language Learner," "ELL"; "Dual Language Learner", "DLL," "bilingual"; 
"Limited English Proficient," "LEP"; "English Speakers of Other Languages," "ESOL." Math searches in 
the ERIC and EBSCO databases included all literature published since 1966. As the science searches 
were intended to update a pre-existing literature synthesis (Lee, 2005), we looked for peer-reviewed 
articles published between 2005 and September 2010. 

To ensure that we had located the key articles in each field, we conducted additional searches of 
specific journals including: Elementary School Journal, Harvard Educational Review, Journal of 
Research in Science Teaching Science Education, Journal of Research in Mathematics Education, and 
the journals published by AERA (e.g.. Educational Evaluation and Policy Analysis). These follow-up 
searches focused on articles published since 2000. Furthermore, as recommended by Ohkee Lee, we 
ran specific Google searches on three authors (L. Khisty, J. Moschkovich, and R. Gutierrez) whose 
work focuses on issues of ELL and math education. We also shared our list of publications with the 
DR K-12 researchers, who offered other possibilities that we investigated. 


^ All journal databases available through EBSCO were searched, including: Academic Search Complete; 
Biomedical Reference Collection: Corporate; Business Source Corporate; EconLit; Environment Complete; 
MEDLINE; SocINDEX; Psychology and Behavioral Sciences Collection. 

^ The asterisk is used to identify results with all possible endings such as education, educating, educator, 
etc. 
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Abstracts for the identified literature were reviewed to select articles meeting the following six 
criteria. The first five are similar to those utilized in Lee's synthesis (2005), while the last one is 
distinct. We did not include literature reviews or conceptual pieces. We looked for studies that 
were: 

1. Published in peer-reviewed journals (conference proceedings were not included). 

2. Directly addressing ELLs in science and/or math education. 

3. Written in English. 

4. Focused on K-12 ELL students. 

5. Empirical, including experimental and quasi-experimental studies; descriptive studies; 
correlational studies; surveys; ethnographic, qualitative or case studies; and studies using 
large-scale achievement data. 

6. Conducted within the United States. 

In total, we identified 63 peer-reviewed articles addressing ELLs and science (40 articles) and/or 
math education (29 articles); these articles were associated with 28 ELL-science and 26 ELL-math 
studies. The full texts of these articles were then reviewed and the characteristics of the studies 
were coded. We then created a matrix similar to the one constructed to capture the characteristics 
of the DR K-12 ELL projects to facilitate comparison between the larger fields of research and the DR 
K-12 ELL portfolio. 

Results 

The results of our study are presented below. We first discuss the expertise of the investigators of 
the DR K-12 projects, then the projects they are conducting, and finally a comparison of the work 
being funded by DR K-12 to research that has been published. The work being conducted by the DR 
K-12 projects and the research published in the literature fall into four domains: student learning, 
instruction, assessment, and curriculum. These domains were represented within both the science 
and math education focused studies. Brief descriptions of the DR K-12 projects included in these 
analyses are included in Appendix A, organized by content area and domain. 

Expertise of DR K-12 Investigators 
Expertise of Pis and co-PIs 

We coded the demonstrated expertise of Pis and co-PIs by looking at the field in which they 
obtained their highest degree, their peer-reviewed published articles, conference papers, and 
grants, and the courses they taught (Exhibit 2). Looking across the proxies, 11 researchers 
demonstrated expertise in ELL/LA in at least one of the proxies. This was less than one-third the 
number of researchers demonstrating expertise in either math or science education (34) or in math 
or science (38). 
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Exhibit 2: Discipiinary Expertise, Aii Researchers (n=41) 


Proxy 

N 

Any Math 
and/or Science 
Education 

Any ELL/LA 

Any Math and/ 
or Science* 

Field of highest degree 

41 

7 

1 

19 

Peer-reviewed articles 

39 

26 

10 

30 

Grants awarded 

36 

26 

7 

28 

Conference papers 

32 

17 

9 

20 

Courses taught 

23 

10 

4 

15 

Any proxy 

41 

34 

11 

38 

* Includes mathematics, science, computer science, and psychology (where not specified as educational 
psychology, such as empirical or cognitive psychology) as well as math and/or science education. 


We observed similar trends among only the projects' Pis (Exhibit 3). While nearly 90 percent of the 
Pis demonstrated expertise in math and/or science education (16), less than a third demonstrated 
expertise in ELL/LA (5). 


Exhibit 3: Discipiinary Expertise, Pis Oniy (n=18) 


Proxy 

N 

Any Math 
and/or Science 
Education 

Any ELL/LA 

Any Math and/ 
or Science* 

Field of highest degree 

18 

3 

0 

7 

Peer-reviewed articles 

16 

14 

4 

14 

Grants awarded 

16 

13 

4 

12 

Conference papers 

12 

6 

4 

6 

Courses taught 

10 

5 

1 

5 

Any proxy 

18 

16 

5 

16 


* Includes mathematics, science, computer science, and psychology (where not specified as educational 
psychology, such as empirical or cognitive psychology) as well as math and/or science education. 


Math and/or Science vs. ELL Expertise in DR K-12's ELL Portfoiio 

The fields of ELL science education and ELL math education are emerging as researchers from 
different backgrounds— math/science education and ELL/LA education— cross domains. To assess 
whether more researchers in the DR K-12 portfolio are entering from backgrounds in ELL/LA or 
math/science education, we looked at the researchers who had demonstrated expertise in one area, 
but not the other. 

It is more common for a researcher to show expertise in either math/science education and/or a 
math/science discipline than only in ELL/LA, suggesting that more DR K-12 researchers conducting 
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work in math or science education and ELL have come from math/science education than from 
ELL/LA research. This trend is the same whether we look solely at Pis or at all investigators. Across 
all researchers, Pis and co-PIs, on DR K-12 ELL projects, 37 demonstrated expertise in math/science 
and/or math/science education without expertise in ELL/LA, while three demonstrated expertise in 
ELL/LA and not in math/science or math/science education (Exhibit 4). 


Exhibit 4: Comparison of Expertise in Math/Science and Math/Science Education and 
ELL/LA, Aii Researchers (n=41) 


Proxy 

N 

Oniy 

Math/Science / 

Math/Science 

Education* 

Oniy ELL/LA 

Field of highest degree 

41 

19 

1 

Peer-reviewed articles 

38 

25 

4 

Grants awarded 

36 

28 

6 

Conference papers 

35 

17 

6 

Courses taught 

33 

15 

3 

Any proxy 

41 

37 

3 


*lncludes mathematics, science, computer science, and psychology (where not specified as educational 
psychology, such as empirical or cognitive psychology). 


When looking at Pis alone, only one researcher has exclusive expertise in ELL/LA, while 15 
demonstrated expertise in math/science or math/science education but not ELL/LA (Exhibit 5). 


Exhibit 5: Comparison of Expertise in Math/Science and Math/Science Education and ELL/LA, 


Pis Oniy (n=18) 

Proxy 

N 

Oniy Math/Science 
or Math/Science 
Education* 

Oniy ELL/LA 

Field of highest degree 

18 

7 

0 

Peer-reviewed articles 

16 

11 

1 

Grants awarded 

16 

12 

3 

Conference papers 

12 

5 

3 

Courses taught 

10 

5 

1 

Any proxy 

18 

15 

1 


*lncludes mathematics, science, computer science, and psychology (where not specified as educational 
psychology, such as empirical or cognitive psychology). 
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Science vs. Math Expertise in DR K-12's ELL Portfoiio 

Our analyses suggested that researchers conducting work with ELLs in math or science education 
were more likely to have a background in math or science than to have a background in ELL/LA. 
However, it is less clear whether more are coming from math or science. Our analysis found that 
across the DR K-12 portfolio of ELL projects, math projects outnumber science projects by four. As 
shown in Exhibit 6, our analysis found that more researchers have their highest degree in a science- 
related field rather than math; however, more have published peer-reviewed articles, presented 
conference papers, been awarded grants, and taught courses in math/math education only than in 
science/science education only. 


Exhibit 6: Comparison of Expertise in Math, Science, and ELL/LA, Aii Researchers (n=41) 


Proxy 

N 

Only Math 

Only Science * 

Only ELL/LA 

Field of highest degree 

41 

6 

13 

1 

Peer-reviewed articles 

38 

15 

8 

4 

Grants awarded 

36 

19 

8 

6 

Conference papers 

35 

12 

5 

6 

Courses taught 

33 

8 

6 

3 

Any proxy 

41 

17 

7 

3 


* For the purposes of this analysis, computer science and psychology are categorizes as "science" 
disciplines. 


Among Pis only, the results were similar. Our analysis found that more researchers demonstrated 
science expertise as measured by highest degree; however, math expertise dominated the 
remaining proxies (Exhibit 7). 


Exhibit 7: Comparison of Expertise in Math, Science, and ELL/LA, Pis Oniy (n=18) 


Proxy 

N 

Only Math 

Only Science * 

Only ELL/LA 

Field of highest degree 

18 

2 

5 

0 

Peer-reviewed articles 

16 

8 

3 

1 

Grants awarded 

16 

10 

2 

3 

Conference papers 

12 

4 

1 

3 

Courses taught 

10 

4 

1 

1 

Any proxy 

18 

8 

2 

1 


* For the purposes of this analysis, computer science and psychology are categorizes as "science" 
disciplines. 


Math, Science, and ELL Expertise Across Teams 

Twelve of the DR K-12 ELL projects listed a PI and at least one co-PI, so that the expertise 
represented on the research team spanned beyond a single researcher. We analyzed the 


Abt Associates Inc. 


Math and Science Education with English Language Learners 


9 


demonstrated expertise resident on the research teams and found that three project teams 
consisted of individuals with demonstrated expertise in all three areas of math and/or science 
education, ELL/LA, and science or math (Exhibit 8). Two projects' teams demonstrated expertise in 
math/science education and ELL/LA but lacked disciplinary math/science expertise. Twelve projects 
did not have key members with demonstrated expertise in ELL/LA and one project did not have 
team members with demonstrated expertise in math/science education. 


Exhibit 8: Expertise within DR K-12 Research Teams (Pis and co-Pis) 


Research Team 

Math/Science Education 

ELL/LA 

Math/Science 

1 

• 


• 

2 

• 


• 

3 

• 


• 

4 

• 



5 

• 


• 

6 

• 



7 

• 

• 

• 

8 


• 


9 

• 

• 


10 

• 


• 

11 

• 

• 


12 

• 



13 

• 



14 

• 



15 

• 



16 

• 

• 

• 

17 

• 


• 

18 

• 

• 

• 


•= Expertise resident in research team of PI and co-PIs. 


For research teams that did not demonstrate expertise at the PI and co-PI level, we then considered 
the expertise represented in the project's advisory groups and affiliated researchers (not identified 
as co-PIs). The additional expertise contributed by advisory groups and affiliated researchers is 
shown with X's in Exhibit 9. These individuals helped fill gaps in expertise on seven projects, but 
eight projects still lacked demonstrated expertise in ELL/LA, one project lacked math/science 
education expertise, and five were missing demonstrated disciplinary math/science expertise. 
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Exhibit 9: Expertise within DR K-12 Research Teams (Pis, co-Pis, and Advisory Groups) 


Research Team 

Math/Science Education 

ELL/LA 

Math/Science 

1 

• 


• 

2 

• 

X 

• 

3 

• 

X 

• 

4 

• 



5 

• 

X 

• 

6 

• 


X 

7 

• 

• 

• 

8 


• 


9 

• 

• 


10 

• 


• 

11 

• 

• 

X 

12 

• 



13 

• 



14 

• 

X 

X 

15 

• 


X 

16 

• 

• 

• 

17 

• 


• 

18 

• 

• 

• 

Expertise resident in research team of PI and co-PIs. 




X= Expertise resident in advisory group or non-co PI research team members. 


Characteristics of DR K-12 Projects 
Focus of Projects 

Slightly more projects focus on math education (7) than science education (11). The most salient 
topics addressed by the projects are instruction (7) and curriculum development (6), following by 
student learning (3), scaling-up (1), and assessments (1). Brief summaries of the DR K-12 projects 
that consider ELL are provided in Appendix A. 

More than half of the projects (10) focus on the middle school years, while eight investigate learning 
in primary school and three in high school (note: four projects addressed both elementary and 
middle school years). While the content for the middle school years is split evenly between the 
subject areas (5 in math, 5 in science), the primary school projects are more likely to be science (5 in 
science, 3 in math) while the high school projects are all focused on Algebra I. 

Science education projects are more likely to focus exclusively on ELLs while the math education 
projects are slightly more likely to study ELLs as a sub-group. In half of the projects (9), either ELL 
students or their teachers are the specific focus; this was true in five science projects and four math 
projects. These projects that focus on ELL specifically involve the primary school level (5) and/or 
middle school (6), while only one targets high school students. The remaining half of the projects 
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focus on groups underrepresented in STEM or on diverse learners and include ELLs as one of the 
multiple sub-groups considered. Of these projects, two are in science and seven are in math; three 
involve students/teachers in primary school, four in middle school, and two in high schools. 

Although ELLs in the United States are a heterogeneous group, for example in the native languages 
spoke, this diversity is not always acknowledged in the project materials. For example, a specific 
language is not identified in most projects; 12 projects did not identify the ELLs' language of origin, 
while four indicated "Spanish" and two identified "Hispanic" students (presumably Spanish 
speakers). 

Objectives 

The majority (14 of 18) of the DR K-12 ELL projects intends to develop, test, and/or refine 
interventions or curricula and related materials or tools in math or science education; one project 
will organize a conference; two are exploratory; and one will produce a book on instructional 
"lessons learned." Six projects contain a technology component and five also emphasize learning 
science through inquiry. As the majority of projects are developing and testing interventions, 
curricula etc., most have research questions that consider the intervention's efficacy in improving 
the outcomes of interest. Three projects consider both efficacy and implementation of these 
interventions, while four others are more exploratory in nature. 

Research Design and Methods 

Projects employ a variety of research approaches (Exhibit 10) and gather data from various sources. 
Eight projects are using experimental or quasi-experimental designs; 10 others are primarily 
descriptive in nature, which is not surprising given that the interventions, materials, etc. being 
studied are typically in early stages of development and do not yet warrant more rigorous 
evaluation designs nor larger-scale testing. Almost all (17) of the projects will use both quantitative 
and qualitative methods. Specific measures that are used include: teacher and student interviews, 
assessment results, classroom observations, teacher logs, and student and teacher surveys. 

All but one project (a conference) will collect data, and eight projects will collect data across 
multiple sites (defined as minimally two school districts; see Exhibit 10); none will utilize an extant, 
nationally representative dataset to answer their research questions. While all projects had not 
selected sites at the time of their proposals' submission, the sites reported collecting data at sites 
that are generally urban (13, 72%) with one science project including a rural community. While sites 
are frequently located in the West, however, no geographic area hosts a majority of the projects. 
The science projects tend to be less regionally diverse than the math projects. Four math projects 
(22%) collect data in more than one geographic region while one project includes sites across the 
country. 

Outcomes 

The projects' proposals describe intended outcomes across three levels: teacher, student, and the 
field overall. For students, the researchers aim to make STEM more accessible and increase their 
participation in the disciplines (see Exhibit 10). While all student projects intend to improve math 
and science skills, content knowledge, and STEM achievement (14), the science education projects 
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are slightly more likely to consider student achievement measures and more than four times as 
likely to analyze English proficiency outcomes as the math projects. 

Several projects will also assess student outcomes including improving attitudes and increasing 
confidence and engagement (6) and literacy (4). Projects are assessing a variety of teacher outcomes 
including changing beliefs and attitudes (4), increasing pedagogical content knowledge (9), and 
improving classroom practice (9). Of the projects that articulate the intent to "expand the field" and 
"make a contribution to knowledge," more focus on expanding knowledge regarding 
underrepresented or diverse student groups (10) than on specifically expanding knowledge 
concerning ELL and math/science education (7). Several want to set standards or principles that 
enable future materials to better serve students. One project seeks to provide a model for 
informal/formal education partnerships. 

Conceptual and Theoretical Frameworks 

The conceptual and theoretical frameworks are not clearly identified in four proposals. Of the 14 
projects for which we could ascertain the project's framework, six utilize general student learning 
models that are not specific to ELLs. For example, one project is guided by the theory that increasing 
teacher knowledge and teacher efficacy-based beliefs improves ELLs' achievement. Another project 
focuses on providing resources for teachers to engage their students (not specifically ELLs) in 
scientific explanation and argumentation to improve their science achievement. Five projects, 
however, discuss frameworks utilizing socio-cultural theories. For example, one states that the 
project understands learning as a social activity situated within a context in which people, cultural 
signs, and tools interact; students develop common ways of thinking and communication as they 
interact over time. Lastly, four science projects (but no math projects; see Exhibit 10) are guided by 
linguistics and semiotics theoretical frameworks. 
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Exhibit 10: DR K-12’s ELL Projects by Content Area 

Study Attribute* Aii Projects Science Math 

(n=18) Projects Projects 

(n=7) {n=ll) 


Research Methods 
Mixed methods 
Research Design 

Experimental or quasi-experimental design 

Descriptive 

Research Scale 

Multiple sites** 

Conducted by research team 

Community Type 

Rural 

Urban 

Geography 

Northeast 

Southeast 

Midwest 

Southwest 

West 

More than one region 
Intended Outcomes 
STEM achievement 
English proficiency or literacy 
Student engagement outcomes 
Theoretical Frameworks 
Linguistic theories 


17 ( 94 %) 

7 ( 100 %) 

10 ( 91 %) 

9 ( 50 %) 

4 ( 57 %) 

5 ( 45 %) 

9 ( 50 %) 

3 ( 43 %) 

5 ( 45 %) 

8 ( 44 %) 

3 ( 43 %) 

5 ( 45 %) 

12 ( 67 %) 

5 ( 71 %) 

7 ( 64 %) 

1 ( 6 %) 

1 ( 14 %) 

0 

13 ( 72 %) 

4 ( 57 %) 

9 ( 82 %) 

5 ( 28 %) 

2 ( 29 %) 

3 ( 27 %) 

2 ( 11 %) 

0 

2 ( 18 %) 

4 ( 22 %) 

0 

4 ( 36 %) 

6 ( 33 %) 

1 ( 14 %) 

5 ( 46 %) 

8 ( 44 %) 

4 ( 57 %) 

4 ( 36 %) 

4 ( 22 %) 

0 

4 ( 36 %) 

14 ( 78 %) 

6 ( 86 %) 

8 ( 73 %) 

4 ( 22 %) 

3 ( 43 %) 

1 ( 9 %) 

6 ( 33 %) 

2 ( 29 %) 

4 ( 36 %) 

4 ( 22 %) 

4 ( 57 %) 

0 


* One project organized a conference and accordingly did not include a research design. 

** Studies that collect data in at least two school districts are considered multi-site projects. 
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Comparison of DR K-12 Portfolio to Published Research 


We compared the work funded under the DR K-12 program to the larger fields of ELL-science 
education and the ELL-math education to explore the unique contribution that the DR K-12 projects 
have the potential to make. Overall, we found that there were more differences between the field 
of ELL science education and the DR K-12 ELL science projects, while the DR K-12 ELL math projects 
were more comparable to the existing efforts in its larger field, as represented by research 
published in peer reviewed journals. 

To identify the research studies in ELL science education, we used the results of a recent synthesis of 
the literature on science education with ELLs (Lee, 2005), and a systematic literature search of the 
research published since 2005. To identify the research studies in ELL math education, we 
conducted a systematic literature search of research going back to 1966. Descriptions of the 
research studies identified through our literature searches are included in Appendix B (science) and 
Appendix C (math), organized by primary research domain— learning, instruction, assessment, 
curriculum, teacher education. 

ELL Science Education 

The DR K-12 ELL science education projects varied from the studies published in the field since 2005 
in that half of the field's efforts focus on ELL science instruction and a quarter on student learning 
while more than half of the DR K-12 projects (57 percent) focus on developing curriculum and 29 
percent focus on instruction (see Exhibit 11). Unlike the DR K-12 science ELL projects, a few of the 
published studies addressed three secondary topics of interest, including scaling-up programs, 
consequences of policy, and most frequently, students' home environment (21 percent). 

We also compared the research designs utilized by the field and DR K-12 to address ELL science 
education topics (Exhibit 12). We made two comparisons: the first with Lee's seminal synthesis 
(informed by articles published between 1982 and 2004), and the second with the studies 
uncovered through our search for research published since 2005. Lee (2005) found that the field 
utilized a range of theoretical and disciplinary perspectives. However, most used qualitative 
methods and tended to produce small-scale, descriptive studies conducted by individual 
researchers; few of the studies used either an experimental or quasi-experimental design. She found 
that few of the studies evaluated the impact of intervention programs on ELLs' science achievement 
and English proficiency, the achievement gaps among linguistic groups, or the results across 
different levels of English proficiency. She also characterized the "sophistication with linguistic 
issues" as "uneven," stating that most did not consider the "complexities inherent in the 
construction of language." 
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Exhibit 11 : Topics Addressed by DR K-12 ELL Science Projects and the Field 

Topic 

Field of ELL Science DR K-12 ELL Science 
Education (n=28) Projects (n=7) 

Primary 



Assessment 

6 (23%) 

1 (14%) 

Curriculum 

2 (8%) 

4 (57%) 

Instruction 

13 (50%) 

2 (29%) 

Pre-service Teacher Education 

3 (11%) 

1 (14%) 

In-service Professional Development 

7 (25%) 

1 (14%) 

Other 

6 (21%) 

0 

Student Learning 

7 (25%) 

0 

Secondary 



Program Scale-Up 

1 (4%) 

0 

Home Environment 

6 (21%) 

0 

Policy 

1 (4%) 

0 


We found that the field has made some progress over the last five years (Exhibit 12), perhaps in 
response to the research agenda Lee put forward in her 2005 synthesis. Our analysis of the research 
published since 2005 suggests that more studies are using mixed (21 percent) or quantitative 
methods (29 percent), although the use of qualitative methods only is still more prevalent (36 
percent). Just as Lee found among studies published before 2005, we found that majority of the 
more recent studies are still descriptive, and only a third use a quasi-experimental design. We did 
see evidence, however, that since 2005 a larger number of studies are being conducted by research 
teams (as indicated by the articles' authors) and include data collected from at least two school 
districts. More studies are using linguistic and discursive theories to frame their research, and 
almost half are now considering measures of student achievement; few however, are considering 
student literacy or engagement outcomes. 

Our comparison of the DR K-12 ELL science projects indicates that the DR K-12 ELL-science education 
projects differ in important ways from those in the studies identified in Lee's published synthesis 
and our updated literature search. A larger portion of the DR K-12 science projects, than the broader 
field, are using experimental or quasi-experimental designs and using mixed methods to answer 
their research questions. The DR K-12 projects are more likely to be analyzing student science or 
math achievement results and more frequently are assessing changes in the ELL students' levels of 
English proficiency. They are also slightly more likely than the field to be investigating student 
engagement. 
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Exhibit 12: Characteristics of DR K-12 ELL Science Education Projects and Larger Fieid * 


Study Attribute 

Field of ELL 
Science 
Education 
to 2004* 

Field of ELL 
Science 
Education, 
2005 to 
Sept 2010 
(n=28) 

DR K-12 
ELL Science 
Education 
Projects 
(n=7) 

Research Methods 




Mixed methods 

rare 

6 (21%) 

7 (100%) 

Quantitative methods only 

rare 

8 (29%) 

0 

Qualitative methods only 

most 

10 (36%) 

0 

Research Design 




Experimental or quasi-experimental design 

rare 

9 (32%) 

4 (57%) 

Descriptive 

most 

18 (64%) 

3 (43%) 

Research Scale 




Multiple sites ** 

infrequent 

7 (25%) 

3 (43%) 

Conducted by research team 

infrequent 

20 (71%) 

5 (71%) 

Intended Outcomes 




STEM achievement 

few 

12 (43%) 

6 (86%) 

English proficiency or literacy 

few 

4 (14%) 

3 (43%) 

Student engagement, agency, empowerment 
outcomes 

rare 

5 (18%) 

2 (29%) 

Theoretical Frameworks 




Linguistic theories 

rare 

15 (54%) 

4 (57%) 


* As described in Lee, 2005. 

** Studies that collect data in at least two school districts are considered multi-site projects. 


We also compared the settings in which published studies and the DR K-12 ELL science education 
projects have and are collecting data (Exhibit 13). DR K-12 projects are more likely to collect data 
from sties in at least two school districts (43 percent vs. 25 percent) oftentimes in the West (57 
percent), while the field is slightly more likely (although does so rarely) to have collected data from 
multiple geographical regions or have utilized nationally representative datasets (7 percent vs. 0 
percent). It also appears that the DR K-12 projects may be more likely to collect data from rural 
communities than the field. 
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Exhibit 13: Setting of DR K-12 ELL Science Projects and the Larger Fieid 


Location 

Field of ELL Science 
Education, 2005 to 
Sept 2010 (n=28) 

DR K-12 ELL Science 
Education Projects 
(n=7) 

Geographical Location 



Nationally Representative Sample 

2 (7%)* 

0 

Northeast 

3 (11%) 

2 (29%) 

Southeast 

5 (18%) 

0 

Midwest 

2 (7%) 

0 

Southwest 

1 (4%) 

1 (14%) 

West 

8 (29%) 

4 (57%) 

More than One Region 

2 (7%) 

0 

Community Setting 



Rural 

1 (4%) 

1 (14%) 

Urban 

19 (68%) 

4 (57%) 


* Totals do not add to 100% as in 7 studies, geographic location of the data collection could not be 
determined. 


ELL Math Education 

We made similar comparisons between the DR K-12 ELL math projects and those of the field (Exhibit 
14). Because there is no synthesis of math education ELL comparable to the Lee synthesis, the single 
comparison for the DR K-12 math ELL projects was to those studies we identified through our search 
for the literature. We found that about half of both published studies field (50 percent) and the DR 
K-12 portfolio (45 percent) are focused on math instruction. The DR K-12 portfolio included more 
studies that are investigating in-service, teacher professional development than the field (36 percent 
vs. 4 percent). Almost a quarter of the field has pursued work related to assessments while none of 
the DR K-12 has. Instead, the DR K-12 projects are more likely to focus on curriculum than the field 
(18 percent vs. 8 percent). The DR K-12 projects also include more projects that are involved with 
scaling up of programs than the field (46 percent vs. 0 percent). 
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Exhibit 14: Topics Addressed by DR K-12 ELL Math Projects and the Field 

Topic 

Field of ELL Math 
Education (n=26) 

DR K-12 ELL Math 
Projects (n=ll) 

Primary 



Assessment 

6 (23%) 

0 

Curriculum 

2 (8%) 

2 (18%) 

Instruction 

13 (50%) 

5 (45%) 

Pre-service Teacher Education 

1 (4%) 

0 

In-service Professional Development 

1 (4%) 

4 (36%) 

Other 

11 (42%) 

1 (9%) 

Student Learning 

5 (19%) 

1 (9%) 

Secondary 



Program Scale-Up 

0 

5 (46%) 

Home Environment 

2 (8%) 

1 (9%) 

Policy 

2 (8%) 

1 (9%) 


Comparing the research designs, we found that the ELL math DR K-12 projects are similar to the field 
in some ways (Exhibit 15). For example, they use experimental or quasi-experimental designs in 
similar proportions. Common outcomes for both are student quantitative achievement measures. 
However, the DR K-12 projects are more likely to use mixed methods (91 percent vs. 23 percent), 
while quantitative only methods were the most popular approach within the field (42 percent). DR 
K-12 math ELL projects were also more likely to examine student engagement outcomes (36 percent 
vs. 8 percent), less likely to consider student literacy outcomes (9 percent vs. 31 percent), and less 
likely to utilize linguistic theories as their studies' frameworks (0 vs. 31 percent). 

We also compared the settings in which the field and the DR K-12 ELL math education projects have 
and are collecting data (Exhibit 16). While the field was much more likely to utilize nationally 
representative extant datasets (19 percent vs. 0), the DR K-12 projects are slightly more likely to 
collect data from at least two school districts (45 percent vs. 38 percent), which were more often 
located in more than one geographical region (36 percent vs. 19 percent). The Western states were 
again the most popular locations for both the field, however, the DR K-12 projects are at least three 
times as likely to be collecting data in the Northeast (27 percent vs. 8 percent), the Midwest (36 
percent vs. 12 percent), or the Southwest (46 percent vs. 4 percent) than the field. The field is more 
likely to collect data from rural communities than the DR K-12 projects (12 percent vs. 0). 
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Exhibit 15: Characteristics of DR K-12 ELL Math Education Projects and the Larger Field 

Study Attribute 

Field of ELL Math 
Education (n=26) 

DR K-12 ELL Math 
Projects (n=ll) 

Research Methods 



Mixed methods 

6 (23%) 

10 (91%)* 

Quantitative methods only 

11 (42%) 

0 

Qualitative methods only 

5 (19%) 

0 

Research Design 



Experimental or quasi-experimental design 

11 (42%) 

5 (45%) 

Descriptive 

15 (58%) 

5 (45%) 

Research Scale 



Multiple sites** 

10 (38%) 

5 (45%) 

Conducted by research team 

16 (62%) 

7 (64%) 

Intended Outcomes 



STEM achievement 

20 (77%) 

8 (73%) 

English proficiency or literacy 

8 (31%) 

1 (9%) 

Student engagement, agency, empowerment 
outcomes 

2 (8%) 

4 (36%) 

Theoretical Frameworks 



Linguistic theories 

8 (31%) 

0 


* One project organized a conference and accordingly is not included in the attribute counts. 
** Studies that collect data in at least two school districts are considered multi-site projects. 
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Exhibit 16: Setting of DR K-12 ELL Math Projects and the Larger Field 

Location 

Field of ELL Math 
Education (n=26) 

DR K-12 ELL Math 
Projects (n=ll) 

Geography 



Nationally Representative Sample 

5 (19%)* 

0 

Northeast 

2 (8%) 

3 (27%) 

Southeast 

4 (15%) 

2 (18%) 

Midwest 

3 (12%) 

4 (36%) 

Southwest 

1 (4%) 

5 (46%) 

West 

7 (27%) 

4 (36%) 

More than one region 

5 (19%) 

4 (36%) 

Community Type 



Rural 

3 (12%) 

0 

Urban 

17 (65%) 

9 (82%) 


* Totals do not add to 100% as in 4 studies, geographic location of the data collection could not be 
determined. 


Conclusions 

CADRE conducted this study to understand the work that is being supported by the NSF's DR K-12 
program to advance our understanding of ELLs' learning in science and mathematics. The study was 
designed to answer three research questions that explore who is being supported by DR K-12, what 
work are they conducting, and what are they contributing to the field. The work consisted of a 
classification of researchers' expertise through a review of CVs, a categorization of their research 
through coding extant project materials, and a comparison of the work funded by the DR K-12 
program to the broader research fields represented by research published in peer-reviewed 
journals, which involved a broad search of the literature and coding of the characteristics of 
published research. Below, the findings from our analyses are used to answer the study's research 
questions. 

What is the expertise of the investigators working on DR K-12's ELL projects? 

Whether we looked across all key investigators or limited the scope to Pis, the DR K-12 investigators 
more often bring expertise in math or science education, or math or science disciplines, than in ELL. 
Similarly, within research teams, expertise in math or science education was most common, 
followed by expertise in a math or science discipline; expertise in ELL was the area most often 
lacking within research teams. 
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What are the key characteristics and objectives of the DR K-12 ELL projects? 


DR K-12 projects are conducting research with ELLs related to science/math instruction, science 
assessment, science/math curriculum, and student learning of science/math. There are slightly more 
math (11) than science (7) projects within the portfolio. More than half of the projects focus on the 
middle school level, than the elementary and high school years. The majority of projects are 
developing and testing curricula or other interventions, thus most have research questions that look 
at the intervention's efficacy. Projects employ a variety of approaches, including experimental or 
quasi-experimental designs (8) and descriptive studies (10). Most projects take a mixed methods 
approach (17), and more than half are being conducted in multiple sites (11). 

How does the DR K-12 portfoiio compare with the work typicai of the iargerfieids of ELL math 
education and ELL science education? 

Our comparisons found differences between DR K-12 projects and the published research for both 
ELL science education and ELL math education, although the differences varied between the 
disciplines. 

DR K-12 ELL science projects compared pretty favorably to the research that has been conducted 
within the broader field, both the pre-2005 literature that was summarized by Lee (2005) and the 
research that has been published since. More than half of DR K-12 science projects are using 
experimental or quasi-experimental designs and using mixed methods to answer the research 
questions as compared to only one-third of research published since 2005. The DR K-12 ELL-science 
projects are more likely to be collecting data from at least two school districts (43 percent vs. 32 
percent). They are also more likely to be analyzing student achievement results (86 percent within 
the DR K-12 portfolio vs. 43 percent in studies published since 2005) and changes in the ELL 
students' levels of English proficiency (43 percent in the DR K-12 projects vs. 14 percent in the 
studies published since 2005). 

Although the DR K-12 ELL math projects were similar along a number of dimensions, they varied in 
some important ways from the broader set of research studies that have been published in the 
journals. Specifically, DR K-12 projects are more likely to use mixed methods (91 percent within the 
DR K-12 portfolio vs. 23 percent in the published literature). In addition, the DR K-12 projects are 
slightly more likely to be designed as a multi-site studies than those in the field (45 percent and 38 
percent, respectively). They were, however, less likely to use linguistic or theoretical frameworks (0 
vs. 31 percent) or include English proficiency outcomes (9 percent vs. 31 percent). 

Discussion 

Our analysis suggests that DR K-12 projects can make a valuable contribution to our understanding 
of math and science education among ELLs, should the projects meet their objectives. This 
knowledge is increasingly important as the ELL population represents a significant and growing 
proportion of U.S. elementary and secondary students. As teams form to investigate related topics. 


Abt Associates Inc. 


Math and Science Education with English Language Learners 


22 




they may consider how they will include expertise in linguistics and language learners. It may also be 
advisable for more projects to consider students' English proficiency as well as measures of student 
engagement and interest in STEM. In doing so, the future research can continue to contribute in 
important ways to the burgeoning fields. 


Abt Associates Inc. 


Math and Science Education with Engiish Language Learners 


23 




References 


American Association for the Advancement of Science. (1989). Science for all Americans: Project 
2061 Report on Literacy Goals in Science, Mathematics, and Technology. Washington, DC: 
American Association for the Advancement of Science. 

Aud, S., Hussar, W., Planty, M., Snyder, T., Bianco, K., Fox, M., Frohlich, L., Kemp, J. Drake, L. (2010). 
The Condition of Education 2010 (NCES 2010-028). National Center for Education Statistics, 
Institute of Education Sciences, U.S. Department of Education, Washington, DC. Retrieved on 
12/29/10, from http://nces.ed.gov/programs/coe/2010/sectionl/indicator05.asp 

Fry, R. (June 6, 2007). How Far Behind in Math and Reading are English Language Learners? Pew 
Hispanic Center Report Washington, DC. Retrieved on 12/29/10, from 
http://pewhispanic.org/reports/report. php?ReportlD=76 

Grigg, W., Lauko, M., and Brockway, D. (2006). The Nation's Report Card: Science 2005 (NCES 2006- 
466). U.S. Department of Education, National Center for Education Statistics. Washington, D.C. 
Retrieved on 12/29/10, from http://nces.ed.gov/nationsreportcard/pdf/main2005/20Q6466.pdf 

National Academies, Committee on Prospering in the Global Economy of the 21st Century. (2007). 
Rising above the gathering storm: Energizing and employing America for a brighter economic 
future. Washington, DC: National Academies Press 

National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2010. 
Rising Above the Gathering Storm, Revisited: Rapidly Approaching Category 5. Washington, DC: 
National Academies Press. Retrieved on 12/29/10, from 
http://www.nap.edu/catalog.php7record id=12999 

National Clearinghouse for English Language Acquisition & Language Instruction Educational 

Programs. (May 2010). The Growing Numbers of English Learner Students: 1997/8-2007/8. The 
George Washington University, Graduate School of Education and Human Development, 
Institute for Education Studies. Washington, DC. Retrieved on December 29, 2010, from 
http://www.ncela.gwu.edU/files/uploads/9/growingLEP 0708.pdf 

Lee, O. (Winter 2005). Science Education with English Language Learners: Synthesis and Research 
Agenda. Review of Educational Research 75(4), 491-530. 

Pearson, W., Jr., & Fechter, A. (Eds.). (1994). Who Will do Science? Educating the Next Generation. 
Baltimore: Johns Hopkins University Press. 

President's Council of Advisors on Science and Technology. (September 2010). Prepare and Inspire: 
K-12 Education In Science, Technology, Engineering, and Math (STEM) for America's Future. 
Retrieved on 12/29/10 from 

http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stemed-report.pdf; 

Riegle-Crumb, C. and King, B. (2010). Questioning a White Male Advantage in STEM: Examining 
Disparities in College Major by Gender and Race/Ethnicity. Educational Researcher 39(9), 656- 
664. 


Abt Associates Inc. 


Math and Science Education with English Language Learners 


24 




Research Studies 


Science and ELL 


Basu, S. J. and Barton, A.C. "Developing a sustained interest in science among urban minority 
youth." Journal of Research in Science Teaching, 44.3 (2007): 466-489. 

Buck, G., Mast, C., Ehlers, N. and Franklin, E. "Preparing Teachers to Create a Mainstream Science 
Classroom Conducive to the Needs of English-Language Learners: A Feminist Action 
Research Project." Vourno/ of Research in Science Teaching, 42.9 (2005): 1013-1031. 

Callahan, R., Wilkinson, L. and Muller, C. "Academic Achievement and Course Taking Among 

Language Minority Youth in U.S. Schools: Effects of ESL Placement." Education Evaluation 
and Policy Analysis, 32.1 (2010): 84-117. 

Chang, M. and Kim, S. "Computer Access and Computer Use for Science Performance of Racial and 
Linguistic Minority Students." Journal of Educational Computing Research, 40.4 (2009): 469- 
501. 

Cho, S. and McDonnough, J.T. "Meeting the Needs of Fligh School Science Teachers in English 

Language Learner Instruction." Journal of Science Teacher Education. 20.4 (2009): 385-402. 

Church, R. B., Ayman-Nolley, S., and Mahootian, S. "The Role of Gesture in Bilingual Education: Does 
Gesture Enhance Learning?" International Journal of Bilingual Education and Bilingualism, 
7.4 (2004): 303-319. 

Ciechanowski, K.M. "'A Squirrel Came and Pushed Earth': Popular Cultural and Scientific Ways of 
Thinking for ELLs." Reading Teacher, 62.7 (2009): 558-568. 

Crawford, T. "What Counts As Knowing: Constructing a Communicative Repertoire for Student 
Demonstration of Knowledge in Science." Journal of Research in Science Teaching, 42.2 
(2005): 139-165. 

Cuevas, P., Lee, O., Hart, J. and Deaktor, R. "Improving Science Inquiry with Elementary Students of 
Diverse Backgrounds." Journal of Research in Science Teaching, 42.3 (2005): 337-357. 

Fang, Z. "The Language Demands of Science Reading in Middle School." International Journal of 
Science Education, 28.5 (2006): 491-520. 

Goldberg, J., Enyedy, N., Welsh, K.M. and Galiani, K. "Legitimacy and Language in a Science 
Classroom." English Teaching: Practice and Critigue, 8.2 (2009): 6-24. 

Langman, J. and Fies, C. "Classroom Response System-Mediated Science Learning with English 

Language Learners." Language and Education: An International Journal, 24.2 (2010): 81-99. 


Abt Associates Inc. 


Math and Science Education with English Language Learners 


25 




Larkin, D. B., Seyforth, S.C. and Lasky, H.J. "Implementing and Sustaining Science Curriculum Reform: 
A Study of Leadership Practices among Teachers within a High School Science Department." 
Journal of Research in Science Teaching, 46 (2009): 813-835. 

Lee, O. "Teacher Change in Beliefs and Practices in Science and Literacy Instruction with English 
Language Learners." Journal of Research in Science Teaching, 41.1 (2004): 65-93. 

Lee, O., Adamson, K., Maerten-Rivera, J., Lewis, S., Thornton, C. and LeRoy, K. "Teachers' 

Perspectives on a Professional Development Intervention to Improve Science Instruction 
among English Language Learners." lourno/ of Science Teacher Education, 19 (2008): 41-67. 

Lee, O., Buxton, C., Lewis, S. and LeRoy, K. "Science Inquiry and Student Diversity: Enhanced Abilities 
and Continuing Difficulties after an Instructional Intervention." Journal of Research in 
Science Teaching, 43.7 (2006): 607-636. 

Lee, O., Deaktor, R., Enders, C. and Lambert, J. "Impact of a Multiyear Professional Development 
Intervention on Science Achievement of Culturally and Linguistically Diverse Elementary 
Students." Journal of Research in Science Teaching, 45.6 (2008): 116-1 AT. 

Lee, O., Lewis, S., Adamson, K., Maerten-Rivera, J. and Secada, W.G. "Urban Elementary School 
Teachers' Knowledge and Practices in Teaching Science to English Language Learners." 
Science Education, 92.4 (2008): 733-758. 

Lee, O. and Luykx, A. "Dilemmas in Scaling Up Innovation in Elementary Science Instruction with 
Nonmainstream Students." Amer/con Educational Research Journal, 42.3 (2005): 411-438. 

Lee, O., Luykx, A., Buxton, C. and Shaver, A. "The Challenge of Altering Elementary School Teachers' 
Beliefs and Practices Regarding Linguistic and Cultural Diversity in Science Instruction." 
Journal of Research in Science Teaching, 44.9 (2007): 1269-1291. 

Lee, O., Maerten-Rivera, J., Penfield, R.D., LeRoy, K. and Secada, W.G. "Science Achievement of 

English Language Learners in Urban Elementary Schools: Results of a First-year Professional 
Development Intervention." lourno/ of Research in Science Teaching, 45.1 (2008): 31-52. 

Lee, O., Mahotiere, M., Salinas, A., Penfield, R.D., and Maerten-Rivera, J. "Science Writing 

Achievement Among English Language Learners: Results of Three-Year Intervention in Urban 
Elementary Schools." Bilingual Research Journal, 32.2 (2009): 153-167. 

Lee, O., Penfield, R. and Maerten-Rivera, J. "Effects of Fidelity of Implementation on Science 
Achievement Gains among English Language Learners." lourno/ of Research in Science 
Teaching, 46.7 (2009): 836-859. 

Luykx, A., Lee, O. and Edwards, U. "Lost in Translation: Negotiating Meaning in a Beginning ESOL 
Science Classroom." Educational Policy, 22.5 (2008): 640-674. 


Abt Associates Inc. 


Math and Science Education with Engiish Language Learners 


26 




Luykx, A., Lee, O., Mahotiere, M., Lester, B., Hart, J. and Deaktor, R. "Cultural and Home Language 

Influences on Children's Responses to Science Assessments." Teachers College Record, 109.4 
(2007): 897-926. 

Lynch, S., Kuipers, J., Pyke, C. and Szesze, M. "Examining the Effects of a Highly Rated Science 

Curriculum Unit on Diverse Students: Results from a Planning Grant." Journal of Research in 
Science Teaching, 42.8 (2005): 912-946. 

Maerten-Rivera, J., Myers, N., Lee, O. and Penfield, R. "Student and School Predictors of High-Stakes 
Assessment in Science." Science Education (2010): 1-26. 

Radinsky, J., Oliva, S. and Alamar, K. "Camila, the Earth, and the Sun: Constructing an Idea as Shared 
Intellectual Property." Journal of Research in Science Teaching, 47.6 (2010): 619-642. 

Rivera Maulucci, M.S. "Resisting the Marginalization of Science in an Urban School: Coactivating 

Social, Cultural, Material, and Strategic Resources." Journal of Research In Science Teaching, 
47.7 (2010): 840-860. 

Robinson, M. "Robotics-Driven Activities: Can They Improve Middle School Science Learning?" 
Bulletin of Science Technology and Society, 25.1 (2005): 73-84. 

Roehrig, G.H., Kruse, R.A. and Kern, A. "Teacher and School Characteristics and Their Influence on 

Curriculum Implementation." Journal of Research in Science Teaching, 44.7 (2007): 883-907. 

Settlage, J., Southerland, S.A., Smith, L.K. and Ceglie, R. "Constructing a Doubt-Free Teaching Self: 
Self-Efficacy, Teacher Identity, and Science Instruction within Diverse Settings." Journal of 
Research in Science Teaching, 46.1 (2009): 102-125. 

Shaver, A., Cuevas, P., Lee, O. and Avalos, M. "Teachers' Perceptions of Policy Influences on Science 
Instruction with Culturally and Linguistically Diverse Elementary Students." Journal of 
Research in Science Teaching, 44.5 (2007): 725-746. 

Siegel, M.A. "Striving for Equitable Classroom Assessments for Linguistic Minorities: Strategies for 
and Effects of Revising Life Science Items." Journal of Research in Science Teaching, 44.6 
(2007): 864-881. 

Spycher, P. "Learning Academic Language through Science in Two Linguistically Diverse Kindergarten 
Classes." The Elementary Schooi Journal, 109.4 (2009): 359-379. 

Watson, S., Miller, T.L., Driver, J., Rutledge, V. and McAllister, D. "English Language Learner 

Representation in Teacher Education Textbooks: A Null Curriculum?" Education, 126.1 
(2005): 148-157. 

Whittier, L. E. and Robinson, M. "Teaching Evolution to Non-English Proficient Students by Using 
LEGO Robotics." Amer/con Secondary Education, 35.3 (2007): 19-28. 


Abt Associates Inc. 


Math and Science Education with Engiish Language Learners 


27 




Young, J.W., Cho, Y., Ling, G., Cline, F., Steinberg, J. and Stone, E. "Validity and Fairness of State 

Standards-Based Assessments for English Language Learners." Educational Assessment, 13 
(2008): 170-192. 

Young, J.W., Steinberg, J., Cline, F., Stone,E., Martiniello, M., Ling, G. and Cho, Y. "Examining the 
Validity of Standards-Based Assessments for Initially Fluent Students and Former English 
Language Learners." Educational Assessment, 15.2 (2010): 87-106. 

Zuniga, K., Olson, J.K. and Winter, M. "Science Education for Rural Latino/a Students: Course 

Placement and Success in Science." Journal of Research in Science Teaching, 42.4 (2005): 
376-402. 


Abt Associates Inc. 


Math and Science Education with Engiish Language Learners 


28 




Mathematics and ELL 


Abedi, J. "Computer Testing as a Form of Accommodation for English Language Learners." 
Educational Assessment, 14.3/4 (2009): 195-211. 

Abedi, J. and Herman, J. "Assessing English Language Learners' Opportunity to Learn Mathematics: 
Issues and Limitations." Teachers College Record, 112.3 (2010): 723-746. 

Beal, C. R., Adams, N.M., and Cohen, P.R. "Reading Proficiency and Mathematics Problem Solving by 
High School English Language Learners." Urban Education, 45.1 (2010): 58-74. 

Brown, C.L. "Equity of Literacy-Based Math Performance Assessments for English Language 
Learners." Bilingual Research Journal, 29.2 (2005): 337-364. 

Callahan, R., Wilkinson, L. and Muller, C. "Academic Achievement and Course Taking Among 

Language Minority Youth in U.S. Schools: Effects of ESL Placement." Education Evaluation 
and Policy Analysis, 32.1 (Mar. 2010): 84-117. 

Cannon, J.E., Fredrick, L.D., and Easterbrooks, S.R. "Vocabulary Instruction Through Books Read in 
American Sign Language for English-Language Learners With Hearing Loss." Communication 
Disorders Quarterly, 31.2 (2010): 98-112. 

Chang, M. "Teacher Instructional Practices and Language Minority Students: A Longitudinal Model." 
Journal of Educational Research, 102.2 (2008): 83-98. 

Chang, M., Singh, K., and Filer, K. "Language Factors Associated with Achievement Grouping in Math 
Classrooms: A Cross-sectional and Longitudinal Study." School Effectiveness and School 
Improvement, 20.1 (2009): 27-45. 

Freeman, B., and Crawford, L. "Creating a Middle School Mathematics Curriculum for English- 
Language Learners." Remedial and Special Education, 29.1 (2008): 9-19. 

Friend, J., Most, R. and McCrary, K. "The Impact of a Professional Development Program to Improve 
Urban Middle-Level English Language Learner Achievement." Middle Grades Research 
Journal, 4.1 (2009): 53-75. 

Ganesh, T.G. and Middleton, J.A. "Challenges in Linguistically and Culturally Diverse Elementary 

Settings with Math Instruction using Learning Technologies." The Urban Review, 38.2 (2006): 
101-143. 

Gutierrez, R. "Beyond Essentialism: The Complexity of Language in Teaching Mathematics to 
Latina/o Students." Amer/con Educational Research Journal, 39.4 (2002): 1047-1088. 

Gutstein, E. "Teaching and Learning Mathematics for Social Justice in an Urban, Latino School." 
Journal for Research In Mathematics Education, 34.1 (2003): 37-73. 


Abt Associates Inc. 


Math and Science Education with English Language Learners 


29 




Han, W. and Bridglall, B.L "Assessing school supports for ELL students using the ECLS-K." Early 
Childhood Research Quarterly, 24.4 (2009): 445-462. 

Khisty, L.L. and Chval, K.B. "Pedagogic Discourse and Equity in Mathematics: When Teachers' Talk 
Matters." Mathematics Education Research Journal, 14.3 (2002): 154-168. 

Kim, S. and Chang, M. "Does Computer Use Promote the Mathematical Proficiency of ELL Students?" 
Journal of Educational Computing Research, 42.3 (2010): 285-305. 

Kinard, B. and Bitter, G.G. "Multicultural mathematics and technology: The Hispanic Math Project." 
Computers in the Schools, 13.1 (1997): 77-88. 

Lee, O., Maerten-Rivera, J., Penfield, R.D., LeRoy, K. and Secada, W.G. "Science achievement of 

English language learners in urban elementary schools: Results of a first-year professional 
development intervention." Journal of Research in Science Teaching, 45.1 (2008): 31-52. 

Lindholm-Leary, K. and Borsato, G. "Hispanic High Schoolers and Mathematics: Follow-Up of 
Students Who Had Participated in Two-Way Bilingual Elementary Programs." Bilingual 
Research Journal, 29.3 (2005): 641-652. 

Martiniello, M. "Language and the Performance of English-Language Learners in Math Word 
Problems." Harvard Educational Review, 78.2 (2008): 333-368. 

Martiniello, M. "Linguistic Complexity, Schematic Representations, and Differential Item Functioning 
for English Language Learners in Math Tests." Educational Assessment, 14.3/4 (2009): 160- 
179. 

Moschkovich, J. "Supporting the Participation of English Language Learners in Mathematical 
Discussions." For the Learning of Mathematics, 19.1 (1999): 11-19. 

Ockey, G. J. "Investigating the Validity of Math Word Problems for English Language Learners with 
DIF." Language Assessment Quarterly, 4.2 (2007): 149-164. 

Watson, S., Miller, T.L., Driver, J., Rutledge, V. and McAllister, D. "English Language Learner 

Representation in Teacher Education Textbooks: A Null Curriculum?" Education, 126.1 
(2005): 148-157. 

Whang, W. "The Influence of English-Korean Bilingualism in Solving Mathematics Word Problems." 
Educational Studies in Mathematics, 30.3 (1996): 289-312. 

Young, J.W., Cho, Y., Ling, G., Cline, F., Steinberg, J. and Stone, E. "Validity and Fairness of State 

Standards-Based Assessments for English Language Learners." Educational Assessment, 13 
(2008): 170-192. 

Young, J.W., Steinberg, J., Cline, F., Stone, E., Martiniello, M., Ling, G., and Cho, Y. "Examining the 
Validity of Standards-Based Assessments for Initially Fluent Students and Former English 
Language Learners." Educational Assessment, 15.2 (2010): 87-106. 


Abt Associates Inc. 


Math and Science Education with Engiish Language Learners 


30 




Zrebiec Uberti, H., Mastropieri, M.A. and Scruggs, T.E. "Check It Off: Individualizing a Math 

Algorithm for Students with Disabilities via Self-Monitoring Checklists." Intervention in 
School and Clinic, 39.5 (2004): 269-275. 


Abt Associates Inc. 


Math and Science Education with English Language Learners 


31 




Appendix A. Description of DR K-12 Projects 



Exhibit A.1 : Descriptions of the DR K-12 ELL Projects 

Primary 

investigator 

Title 

Summary 

Assessment 



Guillermo Solano- 
Flores 

Design and Use of Illustrations 
in Test Items as a Form of 
Accommodation for English 
Language Learners in Science 
and Mathematics Assessment 

Investigates the effectiveness of vignette 
illustrations in test items as a strategy of test 
accommodation for ELL students. 

Curriculum 



Jacqueline Barber 

R&D: The Role of Educative 
Curriculum Materials in 
Supporting Science Teaching 
Practices with English 
Language Learners 

Looks at the effects of educative curriculum 
materials, focusing on teachers of ELL students 

Barbara Crawford 

Fossil Finders: Using Fossils to 
Teach about Evolution, Inquiry 
and Nature of Science 

Develops curriculum through which grade 5-8 
students examine and identify fossils to enhance 
their understanding of the nature of science and 
evolutionary concepts, as well as to motivate them 
to learn more about science 

Martin Gartzman 

R&D: An Architecture of 
Intensification: Building a 
Comprehensive Program for 
Struggling Students in Double- 
Period Algebra Classes 

Designs, develops, and tests the efficacy of student 
and teacher instructional materials and software for 
double-period courses in elementary algebra 

E. Paul 
Goldenberg 

Transition to Algebra: A Flabits 
of Mind Approach 

Creates and tests the effects of instructional 
materials focused on developing conceptual 
understanding and mathematical habits of mind 
that will enable students to succeed in elementary 
algebra 

Beverly Irby 

Collaborative Research: A 
Longitudinal Randomized Trial 
Study of Middle School Science 
for English Language Learners 
(Project MSSELL) 

Consists of a two-year randomized trial evaluation of 
a curriculum model that has been enhanced to 
improved science achievement and academic 
English proficiency of middle school ELL students 
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Exhibit A.1 : Descriptions of the DR K-12 ELL Projects 

Primary 

Title 

Summary 

investigator 

Carolyn Knox 

Collaborative Online Projects 
for ELL Students (COPELLS) 

Studies the effects of linguistically sensitive science 
instructional materials by translating into English, 
enhancing, and testing effects of a series of 
Collaborative Online Projects (COPs) originally 
written in Spanish 

Instruction 

Kathryn Chval 

CAREER: A Study of Strategies 
and Social Processes that 
Facilitate the Participation of 
Latino English Language 
Learners in Elementary 
Mathematics Classroom 
Communities 

Looks at how third grade mathematics instructors 
can better serve the needs of ELLs and develops 
corresponding professional development materials 

Mark Driscoll 

R&D: Fostering Mathematics 
Success in English Language 
Learners 

Studies the effects of the Fostering Geometrical 
Thinking Toolkit (FGTT) program on participating 
middle grades teachers of ELL students 

Catherine Fosnot 

R&D: Project Delta: Digital 
Environments for the Learning 
and Teaching of Algebra 

Extends an existing library of CD-ROM digital 
learning environments by adding an algebra strand 
and evaluating the impact of the new algebra 
materials on elementary teacher development 

Kara Paul Jackson 

SGER: Equity and Access to 
Fligh-Quality Instruction in 
Middle School Mathematics 

Develops tools to appropriately measure equity- 
related aspects of mathematics instruction 

Jennifer Jacobs 

Toward a Scalable Model of 
Mathematics Professional 
Development: A Field Study of 
Preparing Facilitators to 
Implement the Problem- 
Solving Cycle 

Tests the effectiveness of the Problem-Solving Cycle 
model of mathematics professional development 

Kate McNeill 

Supporting Grade 5-8 Students 
in Writing Scientific 
Explanations 

Prepares a book and a research study to investigate 
the impact of that book and accompanying 
professional development on teachers' beliefs and 
classroom practices to support grade 5-8 students in 
writing scientific explanations 
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Exhibit A.1 : Descriptions of the DR K-12 ELL Projects 


Primary 

Investigator 

Title 

Summary 

Patricia Stoddart 

Effective Science Teaching for 
English Language Learners 
(ESTELL): A Pre-Service 
Teacher Professional 
Development Research Project 
Across Three Universities in 
California 

Conducts an experimental study on the impact of 
the ESTELL elementary teacher education program 
for preparing novice teachers to teach science to 
ELLs and a qualitative study on program 
implementation 

Learning 

Carole Beal 

R&D: Closing the Math 
Achievement Gap for English 
Language Learners: 
Technology Resources for Pre- 
algebra 

Studies how ELLs solve word problems and then 
develops online programs to help them solve these 
problems more effectively 

Jeffrey Choppin 

Investigating Equitable 
Discourse Practices in 
Mathematics Classrooms: 
Conference Proposal 

Plan and conduct conference designed to synthesize 
and disseminate research findings on the best ways 
to promote equitable access and opportunities for 
all students to participate, to learn mathematics 
with understanding, and to understand the place of 
mathematics in students' life worlds 

Cathy Kinzer 

Scaling Up Mathematics 
Achievement (SUMA) 

Tests the scalability and replicability of the systemic 
Gadsden Mathematics Initiative 

Karin Wiburg 

Math Snacks: Addressing Gaps 
in Conceptual Mathematics 
Understanding with Innovative 
Media 

Develops and evaluates the efficacy of computer- 
mediated animations and games designed to 
increase student conceptual understanding and 
skills in core mathematical topics of middle grades 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 



Topic Study Participants/Setting Focus of intervention 

Data Coiiection 

Data Analysis 


Science Learning 






Describes 

implementation of an 
after-school program 
focusing on invention 
and exploration 

Basu, S.J. & Barton, A.C. 
(2007) 

Low-income (6th and 7th 
grade), predominantly 
Puerto Rican and Latino, 
who attended a middle 
school in New York City 


Student observations, 
student interviews, 
student work 

Ethnography (qualitative 
analysis) 

Examines impact of at- 
home computer use on 
science learning 
outcomes of students 
from four ethnic 
backgrounds (Caucasian, 
Hispanic, African 
American, Asian) 

Chang, M. & Kim, S. 
(2009) 

Nationally 

representative sample 
(ECLS-K) focusing on 
computer access and 
computer use on science 
achievement of 
elementary school 
students, focusing on 
effects for racial and 
linguistic minorities 

Computer access, 
computer use 

Early Childhood 
Longitudinal Study 
(ECLS-K)'s database, a 
nationally representative 
sample 

Statistical analysis (series 
of regression analyses) 

Examines role of gesture 
when teaching students 
fluent in Spanish but 
with little or no English 
comprehension 

Church, R.B., Ayman- 
Nolley, S. & Mahootian, 
S. (2004) 

51 first grade students in 
Chicago, 26 from 
mainstream English 
speaking classroom, and 
25 from bilingual 
program where students 
speak Spanish only 

Video instruction 
combining speech with 
gesture 

Pre, post student 
assessments 

Quantitative analysis of 
learning tests 


CO 

CT> 


Abt Associates Inc. Math and Science Education with English Language Learners 


Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 



Topic Study Participants/Setting Focus of intervention 

Data Coiiection 

Data Analysis 


Analyzes the text and 
discourse within 
bilingual classes where 
popular culture sources 
of students' science 
understanding are used 

Ciechanowski, K.M. 
(2009) 

2 3rd grade bilingual 
science classes in a 
Spanish/English 
immersion school 


Science textbook, 
popular culture movie, 
classroom observations 

Textual analysis 

Implements and 

Cuevas, P., Lee, 0., Hart, 

Large-scale study of 

PD for instructional 

Elicitation sessions of 

Significant tests of 

assesses a broader 

J., & Deaktor, R. (2005); 

professional 

congruence, inquiry- 

student pre and post- 

means between 

inquiry-based, 
instructional 
intervention intended to 
create greater equity in 
science and literacy 
education for diverse 
elementary students 

Lee, 0., Buxton, C., 

Lewis, S., & LeRoy, K. 
(2006); Lee, 0., Deaktor, 
R., Enders, C. & Lambert, 
J. (2008) 

development 
intervention at six urban 
elementary schools with 
significant numbers of 
culturally and 
linguistically diverse 
students 

based learning; atypical 
lesson where teacher 
relies on more English 
proficient students to 
interpret; educational 
policies 

knowledge assessments, 
student assessments 

pre/posttests 

Assessess professional 

Lee, 0., Mahotiere, M., 

Elementary school 

Professional 

Student writing samples. 

Mixed methods (textual 

development 

Salinas, A., Penfield, R. 

teachers in an urban 

development to increase 

student assessments. 

analysis and HLM, multi- 

intervention (curriculum 

D., & Maerten-Rivera, J. 

school district with a 

science content 

teacher surveys. 

level models) 

and workshops) for 

(2009); Lee, 0., Penfield, 

linguistically and 

knowledge and promote 

classroom observation. 


science teachers in 15 

R., & Maerten-Rivera, J. 

culturally diverse 

inquiry-based science 

scored fidelity of 


urban elementary 
schools 

(2009) 

student population 

instruction 

implementation (based 
on observation) 


Analyzes discourse of 
student's explanation of 
seasonal variation in 
daylight hours in 6th 
grade class 

Radinsky, J., Oliva, S., & 
Alamar, K. (2010) 

6th grade science class in 
a small elementary 
school with an all-Latino 
student population 


Classroom observation 

Case study (qualitative 
analysis) 


CO 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 

Topic 

Study 

Pa rtici pa nts/Setting 

Focus of Intervention 

Data Collection 

Data Analysis 

Science Instruction 






Uses longitudinal data 
from Educational 
Longitudinal Study, 
estimated the effects of 
ESL placement on 
adolescents' college 
preparedness and 
academic achievement 

Callahan, R., Wilkinson, 
L, & Muller, C. (2010) 

More than 15,000 high 
school sophomores 
included in the 
nationally representative 
sample from the 
Educational Longitudinal 
Study 

ESL (language assistance 
programs) placement 

Educational Longitudinal 
Study database 

Statistical analysis 
(descriptive statistics, 
multi-level propensity 
scores, HLM) 

Explores results of 
survey of teachers in a 
community having 
experienced rapid 
growth in its immigrant 
population 

Cho, S., & McDonnough, 
J.T. (2009) 

33 high school science 
teachers of English as a 
Second Language 
students in central 
Virginia 


Teacher survey 

Survey statistical analysis 
(descriptive statistics, % 
respondents, mean, 
standard deviation 

Discusses how reliance 
on use of written 
discourse over other 
practices can results in 
unequal distribution of 
opportunities for 
students to demonstrate 
knowledge 

Crawford, T. (2005) 

1 elementary school 
teacher in Southern 
California with 
significant number of 
students from ethnic 
minority backgrounds 
(Hispanic) 


Student work, classroom 
observations, student 
artifacts (notebooks, 
student writing, student 
projects, homework), 
teacher artifacts, 
student interviews, 
teacher interviews, 
parent interviews 

Ethnography 


CO 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 

Topic 

Study 

Pa rtici pa nts/Setting 

Focus of Intervention 

Data Collection 

Data Analysis 

Describe language of 

Fang, Z. (2006) 

Middle school science 

- 

Science textbooks. 

Textual analysis 

school science (LSS) and 


textbooks and 


researcher experience 


how it differs from 


researchers' experience 




everyday language 


in helping US middle 
school science teachers 




Describes how a teacher 

Goldberg, J., Enyedy, N., 

Spanish in a 6th grade 

- 

Classroom observations 

Case study, discursive 

created an environment 

Welsh, K.M. & Galiani, K. 

science classroom where 



analyses 

for students to use both 

(2009) 

English is the official 




English and Spanish in a 
science classroom 


language of instruction 




Explores the effect of 

Langman, J. & Fies, C. 

1 teacher and 12 

Classroom response 

Field notes, classroom 

Case study, discursive 

classroom response 

(2010) 

students in an 

system (CRS), a 

observations, class 

analyses, time use 

system (CRS) 


intermediate-level 

technology -supported 

debriefing, class focus 

analysis 

intervention on science 


sheltered English science 

intervention 

groups 


teacher's discourse style 


classroom in a urban 

emphasizing dialogic 



in a sheltered English 


high school in the 

inquiry, that supports 



science classroom 


southwest 

user "voting" via 
transmitter 



Implements and 

Lee, 0. & Luykx, A. 

Large-scale study of 

PD for instructional 

Researchers' 

Mixed methods 

assesses a broader 

(2005); Lee, 0, Luykx, A., 

professional 

congruence, inquiry- 

experiences 

(discursive analysis, 
statistical analyses 
(means, standard 
deviations, gain score 
analysis using FILM of 
pre post tests)) 

inquiry-based, 

Buxton, C., & Shaver, A. 

development 

based learning; atypical 

implementing 

instructional 
intervention intended to 
create greater equity in 

(2007); Luykx, A., Lee, 0. 
& Edwards, U. (2008); 
Shaver, A., Cuevas, P., 

intervention at six urban 
elementary schools with 
significant numbers of 

lesson where teacher 
relies on more English 
proficient students to 

intervention, classroom 
observation, teacher 
interviews, teacher 

science and literacy 

Lee, 0., & Avalos, M. 

culturally and 

interpret; educational 

surveys, teacher focus 

education for diverse 

(2007) 

linguistically diverse 

policies 

groups, student 

elementary students 


students 


assessments 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 



Topic Study Participants/Setting Focus of intervention 

Data Coiiection 

Data Anaiysis 


Five-year study of a 
professional 
development 
intervention (curriculum 
and workshops) for 
science teachers in 15 
urban elementary 
schools 

Lee, 0., Adamson, K., 
Maerten-Rivera, J., 

Lewis, S., Thornton, C. & 
LeRoy, K. (2008); Lee, 0., 
Lewis, S., Adamson, K., 
Maerten-Rivera, J., & 
Secada, W. (2008); Lee, 
0., Maerten-Rivera, J., 
Penfield, R. D, Leroy, K., 
& Secada, W.G. (2008) 

Elementary school 
teachers in an urban 
school district with a 
linguistically and 
culturally diverse 
student population 

Professional 

development to increase 
science content 
knowledge and promote 
inquiry-based science 
instruction 

Classroom observation, 
teacher interviews, 
teacher surveys, student 
math and science 
assessments 

Mixed methods 
(statistical analyses and 
content analysis of open- 
ended responses) 

Examines change in 
beliefs/practices of 
elementary teachers 
who shared language 
and culture of their 
students, highlighting 
challenges in 
establishing instructional 
congruence 

Lee, 0. (2004) 

Six bilingual Hispanic 
teachers working with 
4th grade, mostly 
Hispanic students 


Classroom observation, 
teacher interviews 

Qualitative analysis 

Describes the resources 
and strategies used by 
middle school teachers, 
urban fellows, and a 
district science staff 
developer to minimize 
the marginalization of 
science in their school 

Rivera Maulucci, M.S. 
(2010) 

Middle school teachers, 
urban fellows, and 
district science staff 
person in a high-poverty, 
low-performing urban 
school 


Research journal, 
classroom observation, 
teacher interviews 

Critical narrative inquiry 
(ethnography and 
narrative inquiry - 
qualitative analysis) 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 

Topic 

Study 

Pa rtici pa nts/Setting 

Focus of Intervention 

Data Collection 

Data Analysis 

Examines the effect of 
intentional vs. implicit 
approach to English oral 
language development 
in young children 

Spycher, P. (2008) 

39 ethnically and 
linguistically diverse 
kindergarten students in 
2 classrooms with the 
same teacher in an 
urban CA school 

Vocabulary intervention 
utilizing an intentional 
and explicit approach 

Student vocabulary 
assessment via 
elicitation, student 
interviews, teacher 
interviews, classroom 
observations, teacher's 
blog, videotapes of 
lessons, student work 

Mixed methods 
(descriptive statistics, 
two-ANOVA, paired t- 
tests) 

Explores effects of rural 
high school's science 
course placement 
practices on Latino 
student success in 
science 

Zuniga, K. Olson, J.K., & 
Winter, M. (2005) 

157 limited English 
proficiency students in a 
rural high school 

Course placement policy 
whereby students with 
limited English 
proficiency were tracked 
into a science course 
intended for students 
with learning disabilities 

School records (including 
family survey completed 
upon enrollment, GPA, 
and standardized test 
scores in science) 

Statistical analyses 
comparing Latino and 
non-Latino students 

Science Assessment 






Uses state high-stakes 
test results to examine 

Maerten-Rivera, J., 
Myers, N., Lee, 0. & 

Results from state tests 
of 23,854 fifth-grade 

- 

Student state 
achievement tests 

Descriptive statistics and 
multi-level modeling 


factors influencing 
science achievement in 
an urban school district 


Penfield, R. (2010) 


students from 198 
schools in a large urban 
school district with 
diverse student 
population 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 



Topic Study Participants/Setting Focus of intervention 

Data Coiiection 

Data Analysis 


Examines assessments 
for linguistic minority 
students in life science 
courses to make written 
assessments more 
accessible and equitable 
for advanced ELL 
students 

Siegel, M.A. (2007) 

Students in middle 
school life science 
courses in 2 CA schools 
with linguistically and 
culturally diverse 
student populations 


Student assessments 

Statistical analyses 
(regression of pre-post 
test results) 

Examines validity of one 
state's standards-based 
assessments in math and 
science among 5th and 
8th grade students for 
ELL groups 

Young, J. W., Cho, Y., 
Ling, G., Cline, F., 
Steinberg, J. & Stone, E. 
(2008); Young, J.W., 
Steinberg, J., Cline, F., 
Stone, E., Martiniello, 
M., Ling, G., & Cho, Y. 
(2010) 

ELLs and native English 
speakers' results on 
math and science 
standards-based 
assessments in 5th and 
8th grade 


Standards-based math 
and science assessments 
in one state for 5th and 
8th graders in 2005/6, 
state administrative data 

Statistical analyses 
(summary statistics on 
number correct scores, 
internal reliability values, 
standard errors, factor 
analyses) 

Implements and 
assesses a broader 
inquiry-based, 
instructional 
intervention intended to 
create greater equity in 
science and literacy 
education for diverse 
elementary students 

Luykx, A., Lee, 0., 
Mahotiere, M., Lester, 
B., Flart, J. & Deaktor, R. 
(2007) 

Large-scale study of 
professional 
development 
intervention at six urban 
elementary schools with 
significant numbers of 
culturally and 
linguistically diverse 
students 

PD for instructional 
congruence, inquiry- 
based learning; atypical 
lesson where teacher 
relies on more English 
proficient students to 
interpret; educational 
policies 

Student assessments 

Discursive analysis 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 



Topic Study Participants/Setting Focus of intervention 

Data Coiiection 

Data Analysis 


Science Curriculum 


Studies sustained 
implementation and use 
of Integrated Science 
Program (ISP) 

Larkin, D.B., Seyforth, 
S.C. & Lasky, H. J. (2009) 

7 high school science 
teachers using the ISP 
curriculum 

9th grade integrated 
science curriculum, an 
introductory science 
course to help students 
develop necessary skills 
for scientific inquiry 

Department chair 
interviews, teacher 
interviews 

Case study 

Studies the effects of a 
Conservation of Matter 
unit 

Lynch, S., Kuipers, J., 
Pyke, C. & Szesze, M. 
(2005) 

1,500 8th grade students 
in MD middle schools 
with diversity in ethnic, 
linguistic, and socio- 
economic backgrounds 
of students 

"Guided inquiry" 
curriculum 

Student assessments, 
classroom observation 
(videotape) 

ANCOVA and ANOVA if 
pre/posttests and 
ethnography 

Explores robotics 
curriculum used to teach 
middle school physics to 
LEP and ESL students 

Robinson, M. (2005) 

3 8th grade physics 
teachers (one of an ELL 
class, a regular class with 
ELL students, and an 
afterschool class) using 
Robolab 

Robolab (inquiry-based 
learning in robotics- 
physics) 

Teacher interviews 

Case studies (qualitative 
analysis) 

Examines the 
implementation of 
reform-based high 
school chemistry 
curriculum 

Roehrig, G.H., Kruse, R. 
A. & Kern, A. (2007) 

27 high school chemistry 
teachers in a large, 
urban school district 
with a linguistically and 
culturally diverse 
student body 

Inquiry-based, high 
school chemistry 
curriculum 

Teacher interviews and 
classroom observations 

Mixed methods 
(quantitative 
correlations and content 
analysis) 


CO 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 


Topic 

Study 

Pa rtici pa nts/Setting 

Focus of Intervention 

Data Collection 

Data Analysis 

Describes an evolution 
unit around hands-on 
use of LEGO robots to 
improve English 
language writing and 
speaking ability 

Whittier, L.E. & 
Robinson, M. (2007) 

29 students (primarily 
native Spanish speakers) 
in two middle school life 
science classes where 
teachers used Lego 
Robotics 

Inquiry-based use of 
Lego Robotics to teach 
principles of evolution 

Pre and post student 
assessments 

Descriptive statistics 
(mean score comparison 
of pre and post) 

Science Teacher Education 





Explores experiences of 
first-year science teacher 
during school year to 
improve strategies to 
prepare pre-service 
teachers to instruct ELLs 

Buck, G., Mast, C., 
Ehlers, N., & Franklin, E. 
(2005) 

1 beginning teacher of 
middle-level ELLs, a 
science teacher 
educator/ researchers, 
ELL researcher, graduate 
assistant in a 
Midwestern school 


Meeting notes, 
classroom lesson plans, 
classroom observations, 
teacher interviews, 
student interviews, 
student work 

Action research 
(Qualitative analysis) 

Studies pre-service 
teachers during their 
final year of training, 
investigating how to 
develop culturally 
responsive teachers 
through coursework and 
field experiences 

Settlage, J., Southerland, 
S.A., Smith, L.K. & Ceglie, 
R. (2009) 

Pre-service elementary 
education teachers 
participating in student 
teaching placements 

Pre-service teacher field 
placements 

Pre-service self-efficacy 
and belief surveys, pre- 
service teacher 
interviews 

Statistical analyses 


4 :^ 

4 :^ 
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Exhibit B.1 : Science Education and ELL Studies (2005 to September 2010) 

Topic 

Study 

Participants/Setting Focus of intervention Data Coiiection 

Data Anaiysis 

Examines teacher 
training textbooks to 
determine the extent to 
which they discuss 
strategies for teaching 
ELLs 

Watson, S., Miller, T.L., 
Driver, J., Rutledge, V. & 
McAllister, D. (2005) 

25 best-selling textbooks — Textbooks 

Textual analysis 


4 :^ 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 


Topic 

Study 

Pa rtici pa nts/Setting 

Math Learning 



Examines relationship of 
English proficiency and 
math performance in 
high school students 
math performance 

Beal, C. R., Adams, N.M. 
& Cohen, P.R. (2010) 

442 (of whom 209 were 
ELLs) 9th grade students 
in algebra 1 classes in 
four high schools in Los 
Angeles 

Explores the role of an 
NCTM standards-based 
curriculum and teaching 
and learning 
mathematics for social 
justice 

Gutstein, E.(2003) 

7th/8th grade teacher in 
an urban school where 
the vast majority of 
students are Latino and 
low socio-economic 
status 

Explores computer use 
as an educational tool to 
improve math 
achievement for ELL 
students 

Kim, S. & Chang, M. 
(2010) 

4 waves of data for 
students included in the 
Early Childhood 
Longitudinal Study 
Kindergarten Cohort 
(ECLS-K), a database of a 
nationally representative 
sample 


4:^ 


Focus of intervention 


Data Coiiection 


Data Anaiysis 


Web-based tutorial 


Computer use 


State math assessments, 
state English language 
development test, 
student surveys, teacher 
ratings of students 

Participant observations, 
student surveys, school 
records, student 
journals, student 
assessments, student 
work 

ECLS-K, a nationally 
representative database 


Statistical analyses 
(regressions) 


Ethnography 
(practitioner-research, 
textual analysis, 
qualitative analysis) 


Statistical analyses 
(regression analysis, 
HLM) 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 


Topic 

Study 

Pa rtici pa nts/Setting 

Assesses school-related 
attitudes, math 
coursework, and math 
achievement of Hispanic 
high school students 
who had been enrolled 
in bilingual program in 
elementary school 

Lindholm-Leary, K. & 
Borsato, G. (2005) 

139 high school Hispanic 
students in CA who were 
previously ELLs or native 
English speakers who 
had enrolled in a 
bilingual program during 
elementary school 

Assesses English-Korean 
bilingual students 
language difficulties with 
solving math word 
problems 

Whang, W. (1996) 

6 elementary and junior 
high English-Korean 
bilingual students 

Math Instruction 



Explores vocabulary and 
literacy instruction 
techniques for ELL 
students who are 
deaf/hard of hearing 

Cannon, J.E., Fredrick, 
L.D. &. Easterbrooks, S.R. 
(2010) 

4 ELL students who are 
deaf or hard and hearing 
in a 5th grade classroom 


4:^ 

00 


Focus of intervention 


Data Coiiection 


Data Anaiysis 


Two-way bilingual 
program in elementary 
school 


Student surveys, student 
assessments 


Student assessments, 
student interviews 


Statistical analyses 
(mean, standard 
deviations, chi-squared 
tests) 


Case studies (qualitative 
analysis) 


Multimedia materials 


Student elicitations 

(vocabulary 

assessments) 


Case studies (descriptive 
statistics) 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 


Topic 

Examines long-term 
effects of teacher 
instructional grouping 
practices on math 
achievement of language 
minority students 


Investigates the effect of 
within-class grouping on 
math achievement in 
language-minority 
students 


Explores changes in 
teachers' perceptions of 
the ELL student needs, 
effectiveness of 
instructional techniques 
while participating in a 
professional 
development program 


Study 

Chang, M. (2008) 


Chang, M., Singh, K., & 
Filer K. (2009) 


Friend, J., Most, R., & 
McCrary, K. (2009) 


Pa rtici pa nts/Setting 

4 waves of data for 
11,776 ELL and non-ELL 
students from various 
ethnic groups in the 
Early Childhood 
Longitudinal Study 
Kindergarten Cohort 
(ECLS-K), a database of a 
nationally representative 
sample 

4 waves of assessments 
of children from 
kindergarten through 
5th grade included in the 
nationally representative 
sample in the Early 
Childhood Longitudinal 
Study-Kindergarten 
cohort 

70 teachers and their 
235 ELL students in two 
urban middle schools in 
Kansas 


4 :^ 

(O 


Focus of Intervention 


Data Collection 


Data Analysis 


Grouping practices 


Ability grouping 


Professional 
development for 
content-area teachers to 
earn ESL certification 


ECLS-K, a nationally 
representative database 


ECLS-K, a nationally 
representative database 


Student assessments, 
teacher surveys 


Statistical modeling 
(multilevel and general 
linear modeling 
methods) 


Statistical analyses 
(ANOVA, multi-level 
modeling) 


Mixed methods (paired 
t-tests and content 
analysis) 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 


Topic 

Study 

Pa rtici pa nts/Setting 

Investigates how 
technology is in 
mathematics education 
in a Title 1 school 

Ganesh, T.G. & 
Middleton, J. A. (2006) 

1 linguistically and 
culturally diverse multi- 
grade (2nd & 3rd) 
classroom in a Title 1 
school ELL students in 
mixed class 

Explores the role of an 
NCTM Standards-based 
curriculum and teaching 
and learning 
mathematics for social 
justice 

Gutstein, E. (2003) 

1 7th/8th grade teacher 
in an urban school 
where the vast majority 
of students are Latino 
and low-socio economic 
status 

Explores efforts of 
successful instruction of 
Latina/o students in high 
school math 

Gutierrez, R. (2002) 

3 high school math 
teachers who have 
advanced Latino/a 
students in higher level 
mathematics courses 

Examines ELL students' 
learning trajectories 
relative to their school 
environments 

Han, W. & Bridglall, B.L. 
(2009) 

5 waves of data for 
~17,000 for whom 
home-language data was 
available in the Early 
Childhood Longitudinal 
Study Kindergarten 
Cohort (ECLS-K), a 
database of a nationally 
representative sample 


U1 

o 


Focus of intervention 


Data Coiiection 


Data Analysis 


Technology 

Classroom observations, 
teacher interviews, 
school, district, and 
teacher documents (e.g., 
curriculum, model 
lessons, state standards) 

Case study (qualitative 
analysis) 


Classroom observations, 
student surveys, school 
records, student 
journals, student 
assessments, student 
work 

Ethnography 
(practitioner-research, 
textual analysis, 
qualitative analysis) 


Teacher interviews, 
student interviews 

Case studies (qualitative 
analysis) 

School factors 
(instructional services) 

ECLS-K, a nationally 
representative database 

Statistical analyses 
(growth curve modeling) 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 


Topic Study Participants/Setting 


Explores characteristics 
of "talk" in the 
classroom and its impact 
on what students learn 
by analyzing teacher's 
interactions w/ 5th 
grade Latino ELL 
students 

Analyzes discussion 
occurring during a third 
grade math discussion 
on geometric shapes to 
illustrate how one 
teacher supported a 
students' math 
discussion 

Explores a teacher's use 
of individualized self- 
instruction math 
checklists for students 
with learning disabilities 
to help students 


Khisty, L.L. & Chval, K.B. 
( 2002 ) 


Moschkovich, J.N. (1999) 


Zrebiec Uberti, H., 
Mastropieri, M.A., & 
Scruggs, T.E. (2004) 


2 5th grade teachers, 1 
whose Latino students 
have made gains in math 
achievement 


Third grade ELL teacher 


Third grade classroom 
inclusive of ESLand 
students with learning 
disabilities 


U1 


Focus of Intervention 


Data Collection 


Data Analysis 


Mathematical discourse 
(instructional talk) 


Student self-monitoring 
checklist 


Classroom observation 
(including audio and 
videotapes) 


Transcript of a classroom 
lesson 


Student assessments, 
student worksheets, 
student checklists 


Case study (qualitative 
analysis) 


Discursive analysis 


Statistical analyses 
(paired t-tests) 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 
Topic Study Participants/Setting 

Math Curriculum 


Describes the 
development of HELP 
Math, a web-based 
supplemental curriculum 
emphasizing math 
vocabulary for ELL 
students 

Describes the Hispanic 
Math Project field trial of 
a multi-media tutorial 
program that teaches 
measurement 

Math Assessment 

Studies impact of 
accommodations in test 
taking conditions 


Explores relationship of 
ELL status and 
opportunity to learn in 
an algebra course 


Freeman, B. & Crawford, 
L. (2008) 


Kinard, B. & Bitter, G.G. 
(1997) 


Abedi, J. (2009) 


Abedi, J. & Herman, J. 
( 2010 ) 


154 middle school 
students across three 
Colorado school districts 


Hispanic elementary 
students and their 
teachers 


666 ELL students in 4th 
grade (304 were ELLs) 
and 643 8th graders (290 
were ELLs) in a single 
urban school district in 
Southern California 

602 8th grade students 
in southern California 


Focus of Intervention 


Data Collection 


Data Analysis 


Web-based 

supplemental curriculum 
providing sheltered 
instruction in 
mathematical 
vocabulary and 
academic concepts 

Technology-based, 
interactive instructional 
tool 


Computer testing as an 
accommodation for ELL 
students 


Opportunities to learn 
(time) 


Student focus groups, 
teacher focus groups, 
student surveys, teacher 
surveys, student 
assessments 


Expert reviews, student 
assessments, teacher 
interviews, student 
interviews 


Student math 
assessments (NAEP & 
TIMSS), English reading 
proficiency tests, 
student survey 


Teacher surveys, student 
surveys, student reading 
comprehension, and 
student math 
assessments 


Case study (primarily 
qualitative analysis) 


Mixed methods (Chi 
square, t-tests, and 
correlations) 


Statistical analyses 


Statistical analyses 
(HLM) 
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Exhibit C.1 : Math Education and ELL Studies (1966 to September 2010) 

Topic 

Study 

Pa rtici pa nts/Setting 

Focus of Intervention 

Data Collection 

Data Analysis 

Analyzes math test 
scores in Maryland to 
compare ELL 
performance with those 
of fully English proficient 
students 

Brown, C.L. (2005) 

3rd grade ELL and fully 
English proficient (FEP) 
students taking the 
Maryland School 
Performance 
Assessment Program 
tests in 2000 

Literacy-based 
performance 
assessments (LBPA) 
where students read 
multi-level questions 
and explain how to solve 
a math problem with 
words 

Student assessments 

Statistical analyses (t- 
tests, multiple linear 
regressions) 

Examines the linguistic 
complexity in math word 
problems as source of 
differential item 
functioning (DIF) for ELL 
students 

Martiniello, M. (2008) 
Martiniello, M. (2009) 

4th grade MCAS 
(Massachusetts' state 
exam) math exam 
administered in 2003; 24 
4th grade ELL students 
from 6 urban schools in 
Massachusetts 


Student "think aloud" 
interviews, coding of 
math assessments, 
student assessments 

Mixed methods (textual 
analysis, statistical 
analyses, including 
descriptive statistics, 
differential item 
functioning (DIF), 
ordinary Least Squares 
multiple regression) 

Compares ELL and non- 
ELL 8th graders' scores 
on NAEP math word 
problems to determine if 
text operates differently 
for subgroups 

Ockey, G. J. (2007) 

1,174 7th and 8th grade 
students (including 372 
ELL students) who took 
the National Assessment 
of Educational Progress 
(NAEP) in 1992 


Student assessments 

Statistical analyses 
(descriptive analyses, 
differential item 
functioning techniques) 


Ol 
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